Shorten some paths. #4011
Shorten some paths. #4011
Conversation
ClusterFuzz currently passes an unusually long TMPDIR variable into child processes when it's merging coverage. Chromium assumes a reasonably short TMPDIR because it passes it into the UNIX domain socket path within a sockaddr_t, and thus Chromium fails to launch when in this mode. After some investigation as described on https://issues.chromium.org/issues/342197914 it appears as though the less risky solution is to change ClusterFuzz. We could switch to radically different way of figuring out temporary directory paths, but this approach in this PR is just to chop off some characters where it's simple and easy to do so, which should unblock Chromium fuzzers for now.
|
/gcbrun |
docker/base/setup_clusterfuzz.sh
Outdated
| FUZZER_TESTCASES_DISK_FILE=$INSTALL_DIRECTORY/fuzzer-testcases.mnt | ||
| fallocate -l 8GiB $FUZZER_TESTCASES_DISK_FILE | ||
| mkfs.ext4 -F $FUZZER_TESTCASES_DISK_FILE | ||
|
|
||
| # mkfs.ext4 seems to remove the previous allocation, so do it again. | ||
| fallocate -l 8GiB $FUZZER_TESTCASES_DISK_FILE | ||
| mount -o loop $FUZZER_TESTCASES_DISK_FILE $INSTALL_DIRECTORY/clusterfuzz/bot/inputs/fuzzer-testcases-disk | ||
| mount -o loop $FUZZER_TESTCASES_DISK_FILE $INSTALL_DIRECTORY/clusterfuzz/bot/inputs/disk |
There was a problem hiding this comment.
can we call this "ft-disk"?
There was a problem hiding this comment.
Yes - I think I saw a comment from you elsewhere (maybe a chat room?) saying that it might be better to flatten the whole structure a bit. Would you prefer that?
Also I'm happy with any other solution that feeds a shorter TMPDIR to Chromium. If there's some other way, e.g. giving it a TMPDIR inside /tmp instead of in a deeply nested path, that's fine with me - let me know what you'd like me to do.
There was a problem hiding this comment.
Let's just call it ft-disk for now.
There was a problem hiding this comment.
I can't remember why I wanted to get rid of the "inputs" directory but I saw something that made me not want to. I guess it's not super important either way.
|
See test failure. |
Yeah, I think I need your help here. Here's what goes wrong. fuzz_inputs_disk = os.environ['FUZZ_INPUTS_DISK'] # this correctly contains the new shorter path
worker_fuzz_inputs_disk = file_host.rebase_to_worker_root(fuzz_inputs_disk)
with open(os.path.join(worker_fuzz_inputs, 'file'), 'w') as f: # but the directory doesn't exist so this fails
f.write('blah')I can't work out what is supposed to have created |
bot/inputs/ft-disk/README.md
Outdated
| @@ -0,0 +1,4 @@ | |||
| Placeholder for testcases generated by fuzzer. This gets cleared after a task is finished. | |||
There was a problem hiding this comment.
Aha! This is what I missed! It didn't occur to me that there might actually be a fuzzer-testcases-disk directory actually in the ClusterFuzz source tree. Thanks!
This reverts commit c03f2af. This breaks production with the following error: ++ mount -o loop /mnt/scratch0/fuzzer-testcases.mnt /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases-disk 2024-06-07 15:48:16.917 EDT mount: /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases-disk: mount point does not exist. It's not clear why this happens still. My guess is the code that does this was deleted but still exists in the deployed docker image.
This reverts commit c03f2af. This breaks production with the following error: ++ mount -o loop /mnt/scratch0/fuzzer-testcases.mnt /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases-disk 2024-06-07 15:48:16.917 EDT mount: /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases-disk: mount point does not exist. I need to rebuild the docker images before deploying and I don't want to do this on Friday.
ClusterFuzz currently passes an unusually long TMPDIR variable into child processes when it's merging coverage. Chromium assumes a reasonably short TMPDIR because it passes it into the UNIX domain socket path within a sockaddr_t, and thus Chromium fails to launch when in this mode.
After some investigation as described on
https://issues.chromium.org/issues/342197914
it appears as though the less risky solution is to change ClusterFuzz. We could switch to radically different way of figuring out temporary directory paths, but this approach in this PR is just to chop off some characters where it's simple and easy to do so, which should unblock Chromium fuzzers for now.
I can't get
butler.pyto run locally so I'm once again having to test on CI to see what breaks.